ABRA: Online Reading Support. Evaluation Report and Executive Summary

Abracadabra (ABRA) is a 20-week online literacy program composed of phonic fluency and comprehension activities based around a series of age-appropriate texts. Four 15-minute sessions per week are delivered by a teaching assistant (TA) to groups of three to five pupils. This report summarizes the findings of a randomized controlled trial assessing the impact of ABRA on literacy outcomes for Year 1 pupils. The trial also assesses the impact of an offline, paper and pencil version of the same intervention (referred to here at "the non-Information and Communication Technologies (ICT) intervention"). There were 51 participating schools and 2,241 pupils at randomization, and a total of 48 schools and 1,884 pupils were included in the final analysis (84% of the initial pupils at randomization). The trial took place between October 2014 and May 2015. Fifty-one schools were randomly assigned to either receive some version of the intervention or to act as a "control" school delivering business as usual. In the schools receiving the intervention, pupils were randomized to receive one of the following options: (1) ABRA, (2) the non-ICT intervention, and (3) standard literacy provision. The process evaluation involved observing sessions to understand a variety of factors in the intervention. These included an evaluation of which elements contributed to successful implementation, the perceptions and experiences of TAs and project leads, levels of pupil engagement, and the mechanisms behind the estimated impacts. This was an efficacy study, due to the involvement of the developer in the delivery of the program. Key conclusions of this report are: (1) The children who received ABRA, or its offline alternative were found to make two and three months' progress in literacy respectively compared to the children who received standard provision. This positive result would be unlikely to occur by chance; (2) For both ABRA and the offline alternative, the impact for children eligible for free school meals and children with below average pre-test outcomes was larger than for all pupils; (3) Successful implementation contributed to a well-designed and delivered training program which emphasized fidelity and consistency, enforced by ongoing support from the project team; (4) The process evaluation found that both the ICT and non-ICT interventions may be best delivered in groups of similar rather than mixed ability. The process evaluation also suggested minor changes to the intervention to make it more culturally relevant to British pupils, and to remove some repetition in the non-ICT program; and (5) Future research will examine whether ABRA or the non-ICT intervention can be successfully delivered at scale, and will look at longer-term impacts through assessing Key Stage 1 data from this trial.

Publication Title: Education Endowment Foundation ERIC Number: ED581099